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Abstract 

We propose new optimal estimators for the Lipschitz frontier of a set of points. They are 
defined as kernel estimators being sufficiently regular, covering all the points and whose 
associated support is of smallest surface. The estimators are written as linear combina- 
tions of kernel functions applied to the points of the sample. The coefficients of the linear 
combination are then computed by solving related linear programming problem. The 
Li error between the estimated and the true frontier function with a known Lipschitz 
constant is shown to be almost surely converging to zero, and the rate of convergence is 
proved to be optimal. 
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1 Introduction 



Many proposals are given in the literature for estimating a set S given a finite random 
set of points drawn from the interior. This problem of edge or support estimation arises 
in classification (Hardy & Rasson [20]), clustering problems (Hartigan [21j), discrim- 
inant analysis (Baufays & Rasson p]), and outliers detection (Devroye & Wise [8]). 
Applications are found in medical diagnosis (Tarassenko et al [26]) as well as in condi- 
tion monitoring of machines (Devroye & WiSE [S]). In image analysis, the segmentation 
problem can be considered under the support estimation point of view, where the support 
is a convex bounded set in (Korostelev & Tsybakov [2l]). We also point out 
some applications in econometrics (e.g. Deprins, et al In such cases, the unknown 
support can be written 

S^{{x,y): 0< x< 1 ; 0< y < /(x)}, (1) 

where / is an unknown function. Here, the problem reduces to estimating /, called the 
production frontier (see for instance Hardle et al [IB])- The data consist of pair {X,Y) 
where X represents the input (labor, energy or capital) used to produce an output F in a 
given firm. In such a framework, the value f{x) can be interpreted as the maximum level 
of output which is attainable for the level of input x. 

An early paper was written by Geffroy [10] for independent identically distributed ob- 
servations from a density 0. The proposed estimator is a kind of histogram based on the 
extreme values of the sample. This work was extended in two main directions. 

On the one hand, piecewise polynomials estimators were introduced. They are defined 
locally on a given slice as the lowest polynomial of fixed degree covering all the points in 
the considered slice. Their optimality in an asymptotic minimax sense is proved under 
weak assumptions on the rate of decrease a of the density towards by Korostelev 
& Tsybakov [23] and by Hardle et al [19]. Extreme values methods are then proposed 
by Hall et al [16] and by Gijbels & Peng [11] to estimate the parameter a. Estimating 
/ can also been considered as a regression problem Y = f{X) + e with negative noise e. 
In this context, local polynomial estimates are introduced, see Knight [23j, or Hall et 
al [T7] for a similar approach. 

On the other hand, different propositions for smoothing Geffroy's estimator were made in 
the case of a Poisson point process. GiRARD & Jacob [H] introduced estimators based on 
kernel regressions and orthogonal series method [121 US]- In the same spirit. Gardes [H] 
proposed a Faber-Shauder estimator. GiRARD & Menneteau [TB] introduced a general 
framework for studying estimators of this type and generalized them to supports writing 

S = {ix,y): xeE ; 0< y < /(a;)}, 

where / is an unknown function and E an arbitrary set. In each case, the limit distribution 
of the estimator is established. We also refer to Abbar [1] and Jacob & Suquet [22] 
who used a similar smoothing approach, although their estimators are not based on the 
extreme values of the Poisson process. 

The estimator proposed in Bouchard et al [6j can be considered to belong to the inter- 
sect of these two directions. From the practical point of view, it is defined as a kernel 



estimator obtained by smoothing some selected points of the sample. These points are 
chosen automatically by solving a linear programming problem to obtain an estimated 
support covering all the points and with smallest surface. From the theoretical point of 
view, this estimator is shown to be consistent for the Li norm. 

In this paper, we propose several modifications of the above method. First, a bias cor- 
rected kernel is proposed. Second, some regularity constraints are introduced in the 
optimization problem. We show that the resulting estimator reaches the optimal mini- 
max Li rate (up to a logarithmic factor). The estimator is defined in Section [2J Some 
preliminary properties are established in Section El and the main result is presented in 
Section HI Proofs are postponed to Section [51 



2 Boundary estimator 

Let all the random variables be defined on a probability space (fi, J-", P). The problem 
under consideration is to estimate an unknown positive function / : [0, 1] — t- (0, oo) on 
the basis of observations (Xj, Y,i)i=i^,,,^N with independent pairs (Xj, Yi) being uniformly 
distributed in the set 5* defined as 

S^{{x,y): 0<x<l, 0<i/</(x)}. (2) 

Letting 

Cf^ ! f{u) du , (3) 



each variable Xj is distributed in [0,1] with p.d.f. f{-)/Cf while Yi has the uniform 
conditional distribution with respect to Xj in the interval [0, /(Xj)]. In what follows we 
assume / G Il[o,i](/3, i^j^j ), < (3 < 1 that is function / : [0, 1] — )■ (0, oo) is /3-Lipschitz 
with constant : 

\f{x)-f{u)\<Lj^^\x-uf \/x,ue[0,l]. (4) 

The considered estimator : [0, 1] — )■ [0, oo) of the frontier is chosen from the family of 
functions: 



fN{x) = 22^i^h{x,Xi) , Kh{x,t) = K , 



i=l 

a,; > 0, i = l,...,N, 



where is a sufficiently smooth basic kernel function : M — )■ [0, oo) integrating to 
one and having the interval [—1, 1] as its support; the bandwidth parameter G (0, 1/2) 
depends on N such that /i — )■ as X — i- oo; and the function 

(px/h \ 
/ Kit)dt] , a;G[0,l], (6) 
J{x-l)/h J 

corrects the basic kernel K at the boundaries, i.e., when x G [0,h) or x G (1 — /i, 1]. 
Indeed, g{x) = 1 on x E [h,l — h], while g{x) > 1 when x G [0, /i) or x G (1 — h, 1]. 
One may easily observe that 



[ Khix,u)du = l Vxg[0,1] 
Jo 



(7) 



and, consequently, due to interplacing the integral and the derivative, 

/ —Kh{x,u)du = Vxe[0,l]. (8) 
Jo 

Note, that equation ([8]) may be verified directly, as it is demonstrated in the Appendix, 
Subsection I6.1[ 

Denote -ft'max — maxfC(t), (^max — max5f(x), as well as functionals 



C^iv) ^ j Jtf\v{t)\dt, ^eC'{[-l,l]), (9) 

Cp{K,K') ^ g^,^K,^^^Cfs{K) + Cp{K'). (10) 

We also denote by a Lipschitz constant for function : M — )■ M, that is 

\(p{s) — ^p(t)\ < L^\s — t\ with < oo. (11) 

The indicator function is denoted by !{■} which equals 1 if the argument condition holds 
true, and otherwise. 

As it is proved below in Lemma [1] the surface of the estimated support 

Sn = {{x,y) : 0<x<l, 0<y<fNix)} (12) 
may be approximated as follows: 

N 

(13) 



p1 ^ 

/ fN{x) dx = ai + 0(h) . 



This suggests to define the parameter vector a = (ai, . . . , oat)"^ as a solution to the 
following optimization problem 



N 



Jp = minN ai (14) 
subject to 



a 

i=l 



fN{X,)>Yi, t = l,...,N, (15) 

loe 

\U{X,)\<Lf^^g^,^Cp{K,K')^, t = 0,...,N + l, (16) 

N 

^ail{{j -l)/mh< Xi < j/ruh} <Cah, j = l,...,mh, (17) 

1=1 

0<a,, i = l,...,N, (18) 



where parameter rrih is defined to be the integer part of 1/h . This optimization problem 
may be formally written as linear program (LP) 

Jp = min l^a (19) 

a 

subject to 

Y < Aa, (20) 

loe A^ loe A^ 

-Lf^f,g^,^C^{K,K')^lM < Ba < Lf^^g^,^C^{K,K')^l^, (21) 

D^'a < C^hlrn,, (22) 
< a. (23) 



There is one positive parameter Ca in the constraints f|T7j) and fl22|) : its value will be 
discussed in Section HI Moreover, the following notations have been introduced: 



Xq — 0, X^r+i — 1, 



A 



B 4 



\Kh{Xi ,X 



V\H,j=l,...,N 



-^Kh{x,Xi 
ax 



'1 ^^-j) 



x=Xi 



D 
Y 



iJ=l,...,N 

||l{(j - l)/mh <Xi< j/mh}\\i^^^,,,^N-j=i,...,mf, 
(ri,...,F^f. 



(24) 
(25) 
(26) 

(27) 

(28) 
(29) 



3 Preliminary results 

The basic assumptions on the unknown boundary function are: 
Al. < /min < fix) < /max < oo, for all X e [0, 1], 

A2. \f{x) - f{y)\ < Lj^p \x - y\^, for all x,y e [0, 1], with L^^ < oo and < /3 < 1. 

The following assumptions on the kernel function are introduced: 
Bl. : M — 7- [0, oo) has a compact support: supp^g^ -R'(t) = [—1, 1], 

B2. j K{t) dt = 1, 

B3. K is three times continuously differentiable. 

Note, that Qmax = 2 for any unimodal even kernel K{-) meeting conditions B1-B2. We 
quote two preliminary results on the estimator Jn- First, the surface of the related 
estimated support Sn is approximatively ca- Second, the function is Lispchitzian. 
Proofs are postponed to Subsection I5.1[ 



Lemma 1 Suppose Bl, B2 are verified and < h < 1/4. Moreover, let conditions [Tl\ ) 
and ( fl^) hold true for rrih = [h~^\ . Then the surface of the estimated support [W^) meets 
the following inequality: 

- 2Cc,K^^Ji < / fN{x) da; - ttj < 4C„(5(jnax - l)K„,^Ji . (30) 
-^0 i=l 

Remark 1 In fact, only one part of LemmaUl is used in what follows, that is the upper 
bound for the estimator surface given by the right hand side ^3^). 

Remark 2 LemmaUl as well as the further results may be easily extended to basic kernels 
K{-) having also negative values: then i^max = insix\K{t)\, and g{x) > OWx G [0,1] 
should be additionally assumed. 



Lemma 2 Suppose Al and B1-B3 are verified. Let estimator he defined by LP / fT^) - 
l[23\) . Moreover, let h ^ as N ^ oo such that 

log N 

\im-^ = 0. (31) 

Then, there exists almost surely finite N4 = N4^{u) such that for any > A^4 the Lipschitz 
constant for the estimator fj^ over the interval [0, 1] is bounded as follows: 

Lj^ ^ max|A(x)| (32) 

log 

< 2Lf^pg^,^Cp{K,K')^. (33) 

Remark 3 As it can he seen from the proof of Lemma [1[ namely from (f5^)-([57|), one 
might slightly decrease the number of constraints ( 17^) on the estimator derivative pTl)- 
( flgj) . In fact, one could impose those type of constraints not at each point Xi, ? = !,..., A^; 
It would be enough to do at the points with the distance O {{h\ogN / Ny/"^^ between them, 

or at least ((/ilog A^/A^)^/^) in order to keep the same Lipschitz constant for f^ as is 
given by Lemma\^ 

It appears that the estimator f^ being the solution to the optimization problem f|T4|) - 
flTH]) or to its equivalent LP version f lT^ -f l2^ defines the kernel estimator of the support 
covering all the points (Xj, Yi) and, approximately, having the smallest surface, up to the 
term 0{h) specified in Lemma [H Moreover, constraints ( !T6|l - (fT7|) or (l2T]) -f l22|) ensure 
/at G S[o^i](1, Lj^) with a particular Lipschitz constant Lj^ given in Lemma O The 

constraint > for alH = 1, . . . , A^ ensures that fN{x) > for all x G [0, 1] since the 
basic kernel K is chosen to be no n- negative; this seems to be natural for function /(■) is 
positive. Finally, note that the above described estimator ([5]), f|T9|) - fl23|) may be treated as 
the approximation to Maximum Likelihood Estimate related to the estimation family ([S]); 
see Bouchard et a/ [SI [5] for the demonstration. 

4 Main results 

In the following theorem, the consistency and the convergence rate of the estimator to- 
wards the true frontier is established with respect to the Li norm on the [0, 1] interval. 

Theorem 1 Let the above mentioned assumptions Al, A2 and B1-B3 hold true and the 
estimator parameter Ca > 6/max- Moreover, let h as N 00 such that 

log A^ log A^ 

lim inf , , „ > P> , lim ^ , „ ,„ = . (34) 

Then estimator /f3])-/ [^) has the following asymptotic properties: 

||/iv-/||i< (Ci2(/3)/i'' + 2C4(/3)/i-'(logAr/Ar)w) (1 + 0(1)) a.s. (35) 

with 

Ci2(/3) = 2Lf^p (?^ax Cfi{K, K') + AC^{g^^^ - 1)K^..1{I3 = 1} (36) 



and 



C4(/5) ^ 2L 



/,/9 



+ gr.^Cp{K,K') 



^/,/3 / \PJ V 



(37) 



Corollary 1 The maximum rate of convergence which is guaranteed by Theorem U\ 

||/iv-/||i = o((logiV/iV)w) a.s. 
is attained for h meeting the following asymptotics: 

hr^pi^] , 0<p<p-W, (3^ 



which reduces the upper bound ^35\) to 



§_ 

limsupfi^") \\fN-f\\i<C,2{P)p^ + 2C,{P)p~' a.s. (39) 

Let us highlight that f p9|) shows that f^ reaches (up to a logarithmic factor) the minimax 
Li rate for Lipschitz frontier /, see Korostelev & Tsybakov [24j, Theorem 4.1.1. 

Remark 4 The second condition in [34\ ) may be extended to 

log 

hm -^f-— < oo (40) 



which leads to another, more general formula for constants in (35\)-[3l\). 



5 Proofs 

The proof of Theorem [1] which is given in Subsection 15.41 is based on both upper and 
lower bounds derived in Subsection 15.21 and Subsection 15.31 respectively. When proving 
these bounds, we assume that the sequence of the sample X-points (Xj)j=i is already 
increase ordered, without changing notation from Xi to X(j) for the sake of simplicity, 
that is 

X, <X,+i, Vz. (41) 

We essentially apply the uniform asymptotic bound 0(logA^/A^) on AXj = Xj — Xj_i 
proved in auxiliary Lemma |6l Before that, we prove in Subsection 15. li the two preliminary 
results. 

5.1 Proof of preliminary results 

Proof of Lemma [T]. Note that definitions imply the following decomposition. 




Since a, and kernel K are non-negative, it follows that 

/ E "4 ^ (^) d^^t-^JlK (^^^ dx = f:a^ (45) 
and therefore, 

j h{x)dx-Y,(^, < ^ i^max5^«. (/ + J \l{\x-X,\<h}dx (46) 

\0 l-h/ 

/ N 

< {gm..-l)K^..{J2ail{0<X,<2h} (47) 



N 



+ ^ail{l-2h< Xi <1}\ (4J 



1=1 



< {gra..-l)K,^.ACah. (49) 

The inequality (H9|) follows from f|T7|) since both intervals in ( H7|) - fH8|l are of the length 2h 
and thus may be covered by two related intervals of the form [{j — l)/'mh, j /rrih] in f|T71) . 
Consequently, we have proved the upper bound for the difference in the left hand side fH6l) . 
The lower bound is proved in the same manner. Indeed, decomposition fH2l)-( l44l) implies, 
since the term (jH]) is non-negative. 



J Ux)dx-j^a. > -Y^a. [J+J] l^(^) 

/ N 

> -i^max 5^a^l{0<X, </l} (51) 



l+h\ 

dx (50) 



TV 



+ J]aa{l-/i<Xi < 1} (52) 



1=1 

> -K^^^2C^h. (53) 

This completes the proof of Lemma [TJ ■ 

Proof of Lemma [2]. Remind that we assume fj4Tl) . By applying auxiliary Lemma [6] and 
Lemma |8] we first arrive at 

max|/;(x)| (54) 

= max I ft (a;) I (55) 

l<'t<N+l xg[Xi_i,Xi] 



log 1 

< ^/,/3 wC,(ir,K')^ + 8^m^ax^^ 



(X,-X,_0' max |/;"(x) 
xe[Xi_i,Xi] 



(56) 



< ^/,/3^?maxC',(i^,i^0^ + ^C^x^)'mg^|/A^^ (57) 



with Cx > 4C///min- The maximum term in f l57|) is bounded as follows: for any x G [0, 1] 

N 



i=l 



Q3 



N 



< sup 

u,v 

since (see Lemma |5] for the detailed demonstration) 



< h} 



i=l 



sup 

u,v 



Q3 



Kh{v,u) 



where 



Lj^i, — Lk" + 3Li^/_K'max5'max + -^E'fl'max {<iLK + lOi^'^^xfi'max) + Si^^axfi': 

Substituting (jSH]), ([60]) into ([57]) yields 



3 

max ' 



max |/^(x)| < Lj:^^ gr^^y,Cp{K, K')^;j^ + - gmi,^Lj^„Ca ( Cx 



AogN 



under additional assumption (which hold true for all sufficiently large N): 

^ ^ ?>C],C^L^„ logiV 
n> 



(58) 

(59) 
(60) 

(61) 

(62) 

h (63) 
(64) 



(65) 



The result follows. 



5.2 Upper bound for Jn in terms of Jp 

Lemma 3 Let the assumptions of TheoremU\ hold true. Then for any finite 

7 > Lf pg^s,JJp{K) 



(66) 



and almost all u & Q there exist finite numbers Ni = Ni{uj,'~f) such that for all N > Ni 
the LP [W-^M) is solvable and 



J*P<Cf + -fh^ . 



(67) 



Proof of Lemma [31. Consider arbitrary > iVo(ci;) with Nq{uj) from Lemma [B Intro- 
duce function f^{u) = f{u) + 7/1^ and pseudo-estimators 



oti = — - — / J-y{u) du 



HN 



f^{u)du, 2 = l,...,iV (68) 



where 5ij stands for Kronecker symbol. Below we demonstrate that condition f l66|) ensures 
the vector of pseudo-estimators 5 = (5i . . . , Sat)^ to be an admissible point for the LP 
([I9])-(I23]), for any sufficiently large A^. This implies solvability of the LP ([23]) and 



N 



Jp<y^ai= I {f{u) + 7/1^) du = Cf + -fh^ . (69) 
i=i 

Let Cx > 4C///min- For the sake of simplicity, we impose the additional assumptions 

>/3 ^ log^ ^ • //max 1 \ 

< — — < mm <^ , -— \ , 70 

pNh [ 7 pCx ) 

which hold true for all large enough. 

1. First, we prove constraints (fT5|) under ai = cii, i = 1, . . . , N. For arbitrary x G [0, 1], 

N 



fN{x) = Y,^^Kh{x,X,) (71) 

i=l 

= E r AM d.^^-'-'-^-' (72) 

+ ^ / ' /7(«) ^« {Kh{x, Xi) - ir^x, 0)) (73) 

+ ^/" ^(M)dn(ir,(x,X^)-ir,(a;,l)) (74) 



1 



f^{u) Khix,u)du (75) 

/" f ^ ^ / ^^fe(a;,XO + ^fe(a:,A:^-i) . A, 

2^ /AM 5 /s:/,(x,m) ciu (76) 



i=l V 

1 



+ - / f^{u) du {Kh{x, Xi) - Kh{x, 0)) (77) 



1 



+ -/ ^(M)dn(ir,(x,X^)-ir,(x,l)) . (78) 

Now we separately bound each of the summands (I75l) - fl78|) from below. Due to ([7]), the 
main term (1751) is bounded as follows: 

1 .1 

f^iu)Kf,{x,u)du = fix)+^h^+ ifiu)~ fix))Khix,u)du (79) 

Jo 

> f{x) + {^~Lf^^g^^^Cp{K))h^ . (80) 
The i-th summand from (1761) is decomposed and then bounded basing on trapezium 



formula error as follows: 



Xi-i 



Kh{x,Xi) + Kh{x,Xi_i] 



Kh(x,Xi) + Kh(x,Xi_i) 



Kh(x, u) I du 



Kh{x, u) ) du 



x^ 



X,- 



Kh{x,X,) + Kh{x,Xi^i) 



- Kh(x,u] 



du 



> -(/max + 7^^^) max 

12 ue[o,i] 



d^Kh{x,u) 



rXi 

Ljp / \u ~ xf l{\x - Xi\ < 2h} 



du^ 
2h^ 



l{|a; -X,| < 2h} 
[{u - Xi_i) + (X, - u)] du. 



By applying Lemma O the first term is bounded as follows 

> - I Cx^:^ I — [Xi - Xi_i) - Xi\ < 2h}, 



N J 6h^ 
and the second one is bounded by: 



m > 



'-^^;;:''^ Cx^l^i{\x-x^]<2h} 

Moreover, from Lemma [6l one can show first that 

N+l 

5^ l{|x - X,| < 2h}{Xi - <Ah + 



Xi 



Xi- 



\u — x\' au . 



i=l 



N ' 



and second that 



7V+1 



/Xi 
\u-xfdu 

u — x\^du 

log N)/N 
jno- N\ 

< {Ah + Cx- 



< 



x+2h 
x-2h~Cx{logN)/N 

logN 



max 



N I v(i[~2h-Cx{\ogN)/N,2h] 



N J \ N 
Thus, we arrive at the bound for the sum (!76|) as follows: 

logN 



Kh{x,Xi) + Kh{x,Xi^i] , 
u) y ^ Kh{x,u) ] du 



X- 



Nh 



4 + C 



logX\ / Cx/max^i^' logiV 



X- 



Nh 



+ 



6 Nh 



> 



jdmaxCx (^—^rj^^ {Cx fraa.xLK' + '^^'^'^ P ^Lji^Lk)- 



^1) 

(82) 
^3) 

^6) 
(87) 



(89) 
(90) 
(91) 

(92) 
(93) 
(94) 

(95) 



At last, it is similarly demonstrated that both summands fl77|) and fl78l) are bounded above 
by O {{log N / {Nh)Y). For instance, for fl77|) . one obtains 



Xi 



f^{u) du {Kh{x,X,)-Kh{x,0)) 



< {U.. + lh^)Xi\Kh{x,Xi)-Kh{x, 



logA^ 
Nh 



(96) 



Thus, from ([7ID-([96]) it follows for each j = 1, . . . , N that 



fN{X,) > /(X,) + (7 - Lf^^ grnMKW + O 



logN 
Nh 



> Yj (97) 



for sufficiently large > Nq{u) when both inequalities ( 170|) and the following one hold 
true: 



l—Lf^p gma.xC/^{K) > -gmaxCx 



logN 



Cxf max I Lk' + 



12L 



K 



+ 3' 



P 



(98) 



2. Similarly, constraints ( !T6|l hold true under ctj = a^, i = 1, . . . , A^. Indeed, for arbitrary 
X G [0, 1], we now have to bound the absolute value of 



N 



d 



N 



/jvi^) = X] ^ Kh{x, Xi) = ^ai Kh{x, Xi 



i=l 



i=l 



instead of (|7T]l . Here 



~ d 
Kh{x,u) = —Kh{x,u) 
ox 



(see Subsection I6.ip with the following upper bound 

'x — u 



Kh{x,u) 



K' 



h 



~l~ 5'max 



K 



X — u 

h 



(99) 



(100) 



(101) 



deduced form ( 1145p . (I146p . Hence, one may repeat the arguments of (!72 l) -( l78l) by changing 
Kh for Kh. Therefore, all the rates from ( |8T]) - (l971) should be divided by h, while the 
absolute value of the main term of decomposition, due to (IHl), is bounded as follows: 



f^{u) Kh{x,u) du 



' d 
(/(«) - fix)) Kh{x, u) du 



< 



Lf^p gra.xC(s{K,K')h^-' 



(102) 
(103) 



instead of (I7S])-([Hn])- Remind the definition ©-(UnD for Cp{K,K') which follows from 
(IIOII) . Thus, for sufficiently large A^ > A'o(ci;) and for each Xj we arrive at 

/^(X,) < Lf^p gm.xC^{K, K')h^-' + O f < L^^^ gm.xCp{K, i^')^- (104) 



pNh'^' 



Namely, inequality f ll04p holds true almost surely for all those > Nq{u) such that f lTOj) 
is verified and 

i..^M*-.*-0(^-l) > 5,_c.(J^)^ (105) 

where (see Lemma [5] for the detailed demonstration) 

~ ^K' + 5'max-^max , Lj^, = Lk" + Lk' 5'max-^max • (10''') 



3. Finally, the constraints f[T7|) with 

C„ > 6/^ax (108) 

also hold true under = 5j, i = 1, . . . , A^. Indeed, by Lemma Elthe following inequalities 
hold a.s. for all N > Nq{uj) and for each j = 1, . . . , nih, where ruh = [h~^\ : 

5^5a{(j-l)/m,<X, <j7m,} < {f^,^ + ^h^)fl/mf, + 2Cx^;^) (109) 

< 6/^ax/i, (110) 

under additional assumptions (ITOll . Thus, constraints (|T71) are fulfilled under fllOSp almost 
sure, for any sufficiently large A^. 

4. Since all > 0, constraints (fTSjl hold true, and Lemma [3] is proved. ■ 



Remark 5 5y applying Lemma 0, under additional assumptions Ii219\) on h, one may 
ameliorate the related hounds in ( fg^) -(fggl) and ^UBi)-^OE). Indeed, Lemma El being 
applied with its parameter v G (1,2), states that 



TV+l 



^ l{|x - X,| < 2h}{X, - = o ( ^^^^ ) (111) 



i=\ 



h log" N 



hence, the sum of the term [84\ ) is negligible with respect to that of ^8^. It means, roughly 
speaking, that we may remove the term L^' from (E^j, as well as L^, from U06\) . 
However, it does not change much in the main result of the Theorem. That is why we 
restrict ourselves to the pointing out this possibility here. 

5.3 Lower bound for /n 

Lemma 4 Under the assumptions of Theorem [U for almost all u & Q there exist finite 
numbers N2{u)) such that for any x G [0, 1] and for all N > N2{uj) 

2+fi 

/iv(x) > f{x) — — — (112) 



\ N 



with constant Ci{l3) defined in ^^3l\) . 



Proof of Lemma (4]. Let us take use of Lemma [7] and its Corollary [2] introducing 



"y ~ 1,13 5 



fminL j-^p N 



(113) 



Thus, for any > Nq{u) and any x G [0, 1] there exists (with probability one) an integer 
ifc G {1, . . . , N} such that 



and 

Now, the estimation error at a point x can be expanded as 

f{x)~h{x) = [fix) - f{Xj] 

+ [fix,,) -MX,,) 
+ [hixj-Mx) . 

The term in the right hand side (11161) may be bounded as follows 

\fix) - /(X.JI < L^^^ \x - X,f < Lf^^ 5f , 
as well as the term (IllSp 



fNiXiJ - fNix) 



< \x — Xi, I < Lt 5. 

— JN ' I — JJV ■ 



(114) 
(115) 

(116) 
(117) 

(118) 
(119) 
(120) 



with a Lipschitz constant L^^ for the function estimator /Ar(x). Remind that f^iXi,) > 
Yi, due to dUD or (120]). Thus, (ITT5ll implies 



fix,,) - f^iX,,) < iY,, + 5y) - Y,, = 5y . 

Combining all these bounds we obtain from f lll6p that for all N > NQiu) 

fix)-hix)<5y + Lf^,5l + L^J,. 



:i21) 



(122) 

Therefore, applying Lemma [2] and substituting expressions flllSp for 5^ and 5y into fll22p 
lead to the lower bound 



fNix) > fix)-(2Lf^p5l + L^J., 



> fix) 



N 



2+/3 



C4(/3) /logiV\^ 



(123) 
(124) 



for any sufficiently large N (starting from random a.s. finite integer, which does not 
depend on x). The first inequality in has been applied here in order to simplify the 
lower bound. Lemma H] is proved. ■ 



5.4 Proof of Theorem [T] 

1. Since \u\ = u — 2ul{u < 0}, the Li-norm of estimation error can be expanded as 



ll/iv-/||: 



fN{x) - fix) 



dx 



+ 2 



fix) -f Nix) l\fNix)<fix)\dx. 



(125) 
(126) 



2. Applying Lemmas [T] and [3] to the right hand side fll25p yields 

limsup h-^ ( [ \fNix) - fix)] dx] <-f + 4C,((7max " l)i^maxl{/3 = 1} a.s. (127) 
Note, that one may fix 7 = 2Lj^ gmaxCisiK), for instance. 

3. In order to obtain a similar result for the term fll26p . note that Lemma H] implies 

CNix,uj) = elliN) fix) - fNix) < Cii/S) < 00 a.s. 
uniformly with respect to both a; G [0, 1] and > N2iu}), with 

'^BiN) = j^,^-^j . (128) 

Hence, one may apply Fatou lemma, taking into account that ul{u > 0} is a continuous, 
monotone function: 



1 ^ 



limsup e^liN) / fix) - fNix) l\ fNix) < fix) \ dx 

N^oo Jo ^ 

< / limsup (Nix,cu) l{(Nix,uj) > 0} dx 

Jo N^oo 

< Ciil3) < 00 a.s. 



(129) 

(130) 
(131) 



4. Thus, the obtained relations together with fll25p and fll26p imply (!35|) . Theorem [T] is 
proved. ■ 



6 Appendix 



In Subsection 16.11 we establish some properties related to the corrected kernel. Sub- 
section 16.21 presents some auxiliary lemmas which have been used to prove Theorem [H 
Finally, we collect in Subsection 16.31 some lemmas dedicated to the proof of Remark [5l 



6.1 Corrected kernel 

Let the basic kernel function K be defined as in SectionHl and the bandwidth G (0, 1/2). 
Remind the estimator /at defined in ([5]) as follows: 



N 



fNix) = ^aiKhix,Xi) 



1=1 

a,- > 0, i 



[132) 



1,...,N, 



where the kernel function 

Kh{x,t) = h-^K{{x -t)/h) yxe{h,l-h) 

while 



Kh{x,t) = h-'K{{x -t)/h) 



x/h 



K{t) dt 



and 



Kh{x,t) = h-^K{{x -t)/h) 



K{t) dt 



(x-l)/h 



Vx G [0,/i] 



Vx e [1 - . 



(133) 



(134) 



(135) 



Thus, the kernel function Kh{x,t) is defined for any {x,t) G [0, 1] x M, and the estimator 
fll32p is defined for any x G [0, 1] via the kernel Kh{x,t) corrected at the "boundaries". 
One may easily observe that 



/ Kh{x,u)du = l VxG[0, 1] 
Jo 



(136) 



and, consequently, due to exchanging the integral and the derivative, 

^1 Q 



/ —Kh{x,u)du = VxG[0,1] 



(137) 



Note, that equation (11371) may also be verified directly. For instance, on the left boundary, 
i.e. for X G [0, h], we have 



Kh{x, t) = h-^K{{x - t)/h)g{x) 



x/h 



K{t) dt 



Denoting 



we thus have 



d 

Kh{x,u) = —Kh{x,u) 
ox 



and 



9'{x) 



Kh(x,u) 



x/h 



K{t) dt h-^K{x/h) = -g\x)h-^K{x/h) 



h-^K{{x - u)/h)g\x) + g{x)h-'^K\{x - u)/h) 



g{x)h-^ h-^K' 



X — u 



-K 



X — u 



(138) 
(139) 

(140) 

(141) 
(142) 



Hence, the integral 



Kh(x, u) du 



h 



K 



X — u 



h 



du + 



9{x) 
h 



-K 



X — u 



h 



u=l 



u=0 



(143) 



equals zero for a; G [0, /i]. A similar proof might be repeated for x G [1 — /i, 1]. Finally, 
equality fll36p holds true for all a: G (/i, 1 — /i) too, since g[x) = 1 over this interval. 



In what follows, we use more general formulas ([5])-([6]) instead of fll38p . that is 



Khix,t) = h~^K{{x-t)/h)g{x), g{x) = { I K{t)dt 

'{x-l)/h 



X G [0, 1] . (144) 



Therefore, as follows from fll39p . fll44p for any x G [0, 1], 

Kh{x,u) = h-^K{{x -u)/h)g'{x) + g{x)h-'^K'{{x -u)/h) 

gix) f 1 f X — u\ fx — u , , , , 



h \h 



h 



(145) 
(146) 



The following Lemma proves Lipschitz-like constants in fllU7p and (1611) - 

Lemma 5 Let kernel defined in meets the assumptions B1-B3, and the hand- 

width h G (0,1/2). Let Kn be defined by U39\) . Then the following upper bounds hold 



true: 



Kh(x,u) 



^Mx,u) 



Kh{x,u) 



K' ' 



— fi'max^ , 

where L^ = Lk' + -^^k fi'max-^max and L^, = Lk" + Lk' 5'max-^max ■ Moreover, 



dx^ 



Kh(x, u] 



where 



^k" ~ 9max [Lr" + 'iLKiK^g_y,g^s_y, + 3L/^(7inax-^max + ^fi'n 

+ (^i + 2^?Lx^Lx)(l + 2^7m, 



(147) 
(148) 

(149) 



(150) 
(151) 



Proof of Lemma [5l The upper bound f ll47p follows directly from fll45p -f lT^6|) . Further- 
more, taking fll45p . fll46p into account, one easily may come to fll07p since 



■^Kh{x,u) 



n h~^T ~ 
i/max" K ' 



and, similarly. 



^2 



du' 



Kh{x,u) 



^ QmsLxh ^ {Lx" + Lk' fi'max-^max) — fi'max^ ^-^K' 



(152) 



(153) 



Moreover, one may continue calculation of further derivatives from fll39p - fll46l) as follows 



dx' 



Kh{x,u) = g{x)h-Uh-^K' 



+ K 



X — u 

h 

X — u 

h 



{Kh{x,l) - Kh{x,Q)) 



+ g{x)h-^ [h-^K" 



X — u 

h 

X — u 

h 



{Kh{x,l)-Kr,{x,Q)) 



(154) 
(155) 
(156) 
(157) 
(158) 



and 



g3 



X — u 



g^s^h{x,u) = g"{x)h-'\^h-'K'\^—^j (159) 

+ K ( iK,{x, 1) - Kf,{x, 0))] (160) 



+ 2g'ix)h-'[h-'K"(^^j (161) 

+ h-'K' (^^^ {K,{x, 1) - K,{x, 0)) (162) 

+ K(^)(|ir.(.,l)-^K.(x,0))) (163) 

+ g{x)h-' (^h-^'K'" (^^) (164) 

~2 T^n I X — U 



+ h-'K" I — ^ I (Kh(x, 1) - A'ftd, 0)) (165) 
+ (|/4(.,1)-|a-4.,0))(166) 



Moreover, from fll44p the derivatives follow 



g'{x) = 9'ix)h-' (^K (^^^ - K (168) 
g"{x) = 2g{x)g'{x)h-' (k (^^'^ - K (^^)^ (169) 
+ (170) 



therefore, 



|<7'(x)| < gl,^K^,^h~\ (171) 
1/(^)1 < ^/Lx (^i^ + 2 W^Lx) • (172) 

Finally, using the bounds ffT7Tl) -flT72D in ffM -ffWD and the definition of Kh ffM we 



arrive at the bound fl6T|) - fl62l) : 

Q3 



Kh{x,u) 



^ fl'max {Lk + Sfifmax-^max) {^K + fi'max-^^max) 
~^ ^S'max-^max (-^A"' + fl'max -^max 

+ K^^^h^ max u)| ) 

+ Kja^Jt' max 1 9 Kh {x^u) / dx\) 

x,u 

— fl'max 

+ 3Lxfl'max + -K^maxfl'max) 

+ ^maxfl'Lx (3i^max^X + 2 Kj^^x fl'max ) ] 
= fl'max \Lk" + 3Lii-/_K'maxflmax + 3Li^ ^fmax-K^^^x + 3fln 

+ (^?. + 2flLx^Lj(l + 2^7m, 



(173) 

(174) 
(175) 

(176) 

(177) 

(178) 
(179) 
(180) 
(181) 
(182) 



Here we applied the upper bound fll47p as well as the one, followed from fll45p -( inB]) and 

(HMD-dlSSl): 



h max 

x,u 



§^K,ix,u) 



< 



fl'max -^max 



K 



flmax-^max 



) 



fl'max^ {Lr' + Lxg max 



+ -^maxflmax {Lr + -^maxfl'max) ) 



(183) 

(184) 
(185) 



Lemma |5] is proved. 



6.2 Auxiliary lemmas. I 

The following results are proved here for the sake of completeness. 

Lemma 6 Let function / : [0, 1] — )■ M meets the assumption Al and sequence (Xj)j=i^...^Ar 
he obtained from an independent sample with p.d.f. f{x)/Cf by increase ordering Ij^J^, 
where Cf is defined by Ij^. Denote Xq = and Xtv+i = 1. Then for any finite constant 
Cx > 4C///min there exist almost surely finite number Nq = No{u) such that 

log N 

max AXi < Cx^- V X > Xq (186) 

i=l,...,N+l N 

with probability 1. For instance, one may fix constant Cx as follows: 

Cx = 5/max/ /mill • (18''') 

Proof of Lemma [HI Introduce a uniform partition of the interval [0, 1] onto subin- 
tervals with equal Lebesgue measures 

£(Afc) ^ 1/m^ < Cx log N/{2N) , A; = 1, . . . , , (188) 

where size of partition 

= minjinteger m : m > 2X/(Cx logX)} (189) 

2N {2 + e)N , , 

< 1 H < — — - — fl90) 

- CxlogX - CxlogX ^ ' 



for an arbitrary e > and for any sufficiently large N. Hence, the event 

Aj^ = {uj : max AX^ < Cx log A^/A^} (191) 



n 

k=l 



--1,...,N+1 
■ N 

i=l 



(192) 



Basing on Borel-Cantelli lemma we prove that the complementary event A%- = n \ Ajy 
may occur only finite number of times (with probability 1). Evidently, 



N 



k=i \i=i J 
= En 1- / C]'f{u)du] (194) 

1.-1 ,-1 V JAk / 



k=l i=l ^ 

N 



< m^{l-^-K^i)j (195) 

i2 + e)N { U,^Cx , 

= O fA^^"-^'"'"^-^/^(^+^^'^/)') . (197) 



Hence, condition Cx > 4C///min implies the existence of positive e ensuring the conver- 
gence of series 

oo 

J2 P {^n) < OO , (198) 



N=l 



and the Borel-Cantelli lemma applies. Note, that events Hili {-^i ^ ^fc) ^'^ ^^^^ depend 
on renumbering of (Xj)j=i ... tv which lead to fll94p from fll93p : moreover, we have used 
both definition (I189p and inequality 1 — a; < there in fll95l) - fll96p . Lemma E] is proved. 



Lemma 7 Let random sample {(Xj,Fj) | i = 1, . . . , A^} he defined as in Section\E Let 
sequence 6^ = Sx{N) be positive, and for some e > 

liminf > 0. (199) 

Define 

nig = minjinteger m : m> 5^^} (200) 

and assume a positive sequence 6y = Sy{N) < /^in meeting for all sufficiently large N the 
inequality 

S>nms^—, with s ■ (201) 

Then, under the assumptions of Lemma 0, with probability 1, there exists finite number 
Nq{u) such that for any N > Nq{u) there is such a subset of points {(Xj^,, YiJ , k = 1, . . . , m^} 
in the sample {{Xi, Yi) , i = 1, . . . , N}, that the following inequalities hold: 

ik-l)/ms<X,^<k/ms, /(X,J - 5 < F,, < /(X,J . (202) 



Proof of Lemma [T], It is similar to that of Lemma |6l Introduce an equidistant partition 
of the interval [0,1] onto subintervals [{k — l)/m^,k/m^, k = l,...,mg. Moreover, 
introduce the related subsets in 

Afc ^ {{u, v):{k- l)/ms <u< k/m, , f{u) - 6^ < v < f{u)} , A; = 1, . . . , . (203) 

Hence, the event 

^ {cu : yk = l,...,ms3t = l,...,N ■.{X,,Yi) eAk} (204) 

N 



n 

k=l 



[j{{X„Y,)eA,} 



.1=1 



(205) 



Basing on Borel-Cantelli lemma we prove that the complementary event A% = n \ Aj^ 
may occur only finite number of times (with probability 1). Evidently, 



N 



k=l \i=l ) 

= En^-/ Cj'f{u)dudv] (207) 

< (208) 

< (1 + 5.-^) exp I log ivj (209) 
= O (^A^i-^-/--'^/^/ j . (210) 

Hence, condition k > (2 — e)Cf/ f^am implies 

oo 

J2p{A%)<oo, (211) 

N=l 

and one may apply Borel-Cantelli lemma. Lemma [7] is proved. ■ 

Corollary 2 Let 6x and 6y meet the conditions of Lemma Then, with probability 1, 
for any N > Nq{uj) and any x G [0,1] there exists integer ik G {1,...,A^} such that 
\x - X,, I < 4 and /(X,J -5y< Y,, < /(X, J . 

Lemma 8 Let function (7 : [0, A] — )• M be twice continuous differentiable, A > 0. Then 

A^ 

max \g{x)\ < max{|(?(0)|, |(?(A)|} + — max \g'\x)\ . (212) 

xG[0,A] O a;G[0,A] 

Proof of LemmaO Denote g^^ = max{|5f(0)|, |(7(A)|}. It suffices to prove the case where 
a point xi G (0, A) exists with 

\g{xi)\=ma.x\g{x)\>gi,. (213) 

xe[o,A] 



Then g'{xi) = 0, and for any x G [0, A] 



g{xi)=g{x)~ I dt j g"{u)du. (214) 

J XI J X\ 



Therefore, putting a; = A one obtains from (I214p 



\gix,)\<\g{A)\+ [ dt f |^»|cin<^, + ^^— ^ max (215) 

Jx^ Jx, 2 XG[0,A] 



' Xl J Xl 

Similarly, fixing x = there in (I214p leads to 



\g{xx)\ < \gm + dt \g"{u)\du <g, + ^ max \g"{x)\ . (216) 

Jo Jxr 2 x€[0,A] 

Thus, combining (1215^ and ( 1216^ we arrive at 

\g{xi)\<g, + lmm{{A-Xif,xl} meix \g"{x)\. (217) 

2 a;G[0,A] 

Since 

A^ 

max min{(A — a;)^, x^j = — , (218) 

xe[o,A] 4 

the desired inequahty fl2T2l) follows immediately from f l2T3D . fl2T7D- fl2T8D . ■ 



6.3 Auxiliary lemmas. II 

Lemma |9] states the results announced in the Remark [5l Subsection [ST 



Lemma 9 Let numbers form a positive sequence, non-increasing for N > Ni and 
meeting condition 

^<1 + ^ ViV>iVi (219) 
N 

with finite, positive constants k and Ni and such that 

log 

lim = . (220) 

Then, under the assumptions of LemmalB, as N ^ oo, for an arbitrary u > 1 and for 
any x G [0, 1] 

Sj, ^ Y.iAX.f l{\x - X.I < 2h^} = ( ^"^^^f ^ ) a.s. (221) 

i=l ^ ^ 

where o(-) does not depend on x. 
Proof of Lemma [91. 

1. Remind that the sequence of random points (Xj) is obtained from that of i.i.d. with 
the p.d.f. f{-)/Cf for Xj by their increase ordering. Furthermore, AXj = Xj — Xj_i, 
Xo = 0, and Xat = 1. Introduce a-algebras J^n = • • • ,XAr}. Thus, {Sn,J^n) is a 

non-negative stochastic sequence. Let us denote X the new point (hence, independent 



of J-7v) when passing from Sn to Sn+i- With these notations and due to the evident 
inequahty 

l{\x - X,\ < 2hN+i} < - X,| < 2/i^} 



one may write 

EiSM+ilJ'N) < e\J2i{x e[x^.i,x^)} 



' N+l 



J = l 



'N+l 



J2i^Xi)H{\x - X,| < 2hN} (222) 



+ {{X - Xj_i)H{\x -X\< 2hN} 



+ {Xj - Xfl{\x -Xj\< 2hN}) 



N 



(223) 



(224) 



N+l 



< 5^-^l{|x-Xj| <2/ijv}E{1{Xg [Xj_i,Xj)} (225) 
• [(AX,)3 - (X - X,_i)' - {X, - X)3] I J-^} (226) 

Af+l 

+ Y,^{iX -Xj-i?HX e [X,_i,X,)}(l{|x-X| < 2/i^}(227) 

- Xj\ < 2hN}) \J^n} ■ (228) 
2. The first need now is to evaluate the conditional expectation in fl225p -f E2^ . A simple 



algebras imply 

{AXjf - (X - Xj_i)^ - {Xj - Xf = 3{X{Xj + Xj_i) - X^ - XjXj,i)AXj 

which is non-negative for any X G [Xj_i,Xj). Therefore, the bounding from below leads 
to 

E {1{X e [X,_i, X,)} [(AX,)^' - (X - X,_i)3 - (X, - Xf] (229) 



SAX, / (x(X, + X,_0 -x'- X,X,_i) dx 



mm / A -1^ \4 



> 



2C 



f 



(AX,)^ 



(230) 
(231) 



Substituting to (I225p - (l226p and applying lensen's inequality for the convex function 



S 



N 



i^{\x~X,\<2h^} 



^ Ef=i - X,\ < 2/^^} (AX,)' 



< 



#{|x-X,-| <2/i;v} 



lead to 



where 



E(5^|^^_0 < S^^^ - ^ ^^^^ + 



1 ^ 

- 5^ l{|a; - X,| < 2hN^,} = Oih^^i) , 



(232) 



(233) 



(234) 



and Tat denotes the related sum in fl227p - fl228l) . that is 

N 

A ^E{(X-X,„i)3l{XG [X,_i,X,)}(l{|x-X| <2/i^_i} (235) 

- X,| < 2/i^_i}) I . (236) 

Note, that one may define 0/0 = to treat the case of zero denominators there in (12321) . 
for instance. 

The bound 0{hjs[) for gjv stated in (12341) is proved below in Lemma [TTl In order to 
bound r^r from above one may easily see that the difference between the two indicators 
in (I235l) - fl236p is positive iff the first of them equals 1 while the second does 0. Due to 
Lemma [6] and the property (12201) . i.e. log N/{NhN) 0, this may almost surely arise 
only for the following event (for any sufficiently large N): x — 2/iAr_i < Xj_i < X < 
X + 2/ijv-i < Xj . Given a sequence (Xj), this event arises only for one j, say j = Jq, 
which depends on x . Thus, 

TN < r\u-X,^_,fl^du<^-^ max (X, - X,_0' (237) 

Cf Cf i=l,...,N+l 

= 0[{\ogN/Nf), (238) 
with non-random O(-) being independent of x, from Lemma El 

3. The next step is to come from the nonlinear inequality (12331) to a linear one. The 
convexity of function il){s) = s^^^ gives, for an arbitrary a^v > 0, the lower bound as 
follows: ^ ^ 

Thus, inequality (I233P and the choice un-i — ci^qN /N"^ with a > lead to 



nsN\:Fr,-,) < - ( ) + 1^ V^' + (239) 



'iCf \NqNj 6Cf \Nq_ 



where 



In 

< S^.,-^Sr,-^ + ^^qN + rN (240) 
/i^^>2 + «: (241) 



for sufficiently large a. 

4. Finally, using relations (I234p -( !236l) in ( 1239p -( l240l) and applying Lemma [10] we arrive 
at the result of Lemma M ■ 

Lemma 10 Let [w^jJ-'n) be non-negative stochastic sequence meeting the inequality 

E{wr,\J'N-i)<{l-j^)w^^^ + ^^ a.s. VX > X^ (242) 

where > p + k, and k meet conditions of Lemma \^ d^ is J^N^i-measurable, non- 
negative and bounded a.s., and Xi < oo. Then, as N ^ oo, for any u > 1 

wn = o{ 1 a.s. (243) 



Proof of Lemma IIOI Introduce 



The inequalities logA^ > log(A^ — 1), hj^^i > h^, and fl242p imply 



EKI^^.O < 1-^ ^-^-i + TFT^TTF (245) 



Since 

f2 ^~l'^ = ^ and y^^f^<oo a.s., (247) 
^ N ^ N log'' ^ ' 

one may apply Robbins-Siegmund almost supermartingale convergence theorem ROBBINS 
& SlEGMUND [25] which implies f at — as — t- oo. Lemma [TO] is proved. ■ 

Lemma 11 Let h = hf^r ^ as N ^ oo. Then, under the assumptions of Lemma\^ and 
Lemma\^ the hound l[234\) holds true for any x G [0, 1], that is 



1 ^ 

gTV = ^ 5^ l{k - X,| < 2/i^_i} = Oih^) (248) 



where O(-) does not depend on x. 

Proof of Lemma llll Introduce 

= l{|x - X,| < 2/ijv-i} - P{\x - Xi\ < 2/ijv-i} (249) 
leading to the decomposition 

1 ^ 

qN = P{\x-Xi\<2hj^^i} + -J2Q- (250) 

i=i 

Since Xj are i.i.d. with the bounded p.d.f. f{-)/Cf, the probability 

P{\x-Xi\<2hN-i}< ^du = OihN-i) (251) 

Jx-2hM-i 

with O(-) being independent of a; and of i. Furthermore, observe that \Q\ < 1 a.s., and 

EQ = 0, EC^ <P{\x-Xi\<2hN-i} = 0{hN^i). (252) 

Thus, in order to bound the stochastic term in the right hand side (12501) one may apply 
the Bernstein inequality (see, e.g., BiRGE & Massart |,3j or BoSQ [4], Theorem 2.6) 
with the standard treatment via Borel-Cantelli lemma (e.g., as in Bouchard et a/ [5], 
Appendix, Lemma 5). This directly yields 

= 0{hr,) + 0\[ ^^t^^ ^ ^ I = Oikj^) a.s. (253) 



N 



Lemma [TT] is proved. 
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